Search for: All records

Abstract Teleoperation can enable human intervention to help handle instances of failure in autonomy thus allowing for much safer deployment of autonomous vehicle technology. Successful teleoperation requires recreating the environment around the remote vehicle using camera data received over wireless communication channels. This paper develops a new predictive display system to tackle the significant time delays encountered in receiving camera data over wireless networks. First, a new high gain observer is developed for estimating the position and orientation of the ego vehicle. The novel observer is shown to perform accurate state estimation using only GNSS and gyroscope sensor readings. A vector field method which fuses the delayed camera and Lidar data is then presented. This method uses sparse 3D points obtained from Lidar and transforms them using the state estimates from the high gain observer to generate a sparse vector field for the camera image. Polynomial based interpolation is then performed to obtain the vector field for the complete image which is then remapped to synthesize images for accurate predictive display. The method is evaluated on real-world experimental data from the nuScenes and KITTI datasets. The performance of the high gain observer is also evaluated and compared with that of the EKF. The synthesized images using the vector field based predictive display are compared with ground truth images using various image metrics and offer vastly improved performance compared to delayed images. 

Estimating the position of the bucket or tool on an agricultural/construction vehicle is becoming increasingly important to enable operator assistance such as automation of repetitive movements. Such end-effector position estimation is normally done through measurement of individual actuator’s movements inside kinematic linkage mechanisms that move the end-effectors. This paper develops an alternate inertial measurement unit (IMU) based end-effector position estimation system that offers significant advantages of low cost and easy installation. An IMU located on a rotating linkage in a mechanism is used to estimate the angular motion of the linkage. Key challenges arise from the fact that the accelerometer signals of the IMU experience significant disturbances from dynamic accelerations and from vehicle and terrain-induced vibrations. First, an adaptive feedforward algorithm is used to remove the influence of vibrations on the accelerometer signals. Then a nonlinear observer is utilized to combine accelerometer and gyroscope signals and reject the influence of vehicle accelerations. Experimental results are presented from a laboratory test rig and preliminary experimental results from a full-scale tracked skid steer loader vehicle. The results show that an accuracy better than 1 degree in linkage orientation estimation is achieved in the presence of vibration disturbances. 

Teleoperation is increasingly used in the operation of delivery robots and is beginning to be utilized for certain autonomous vehicle intervention applications. This paper addresses the challenges in teleoperation of an autonomous vehicle due to latencies in wireless communication between the remote vehicle and the teleoperator station. Camera video images and Lidar data are typically delayed during wireless transmission but are critical for proper display of the remote vehicle's real-time road environment to the teleoperator. Data collected with experiments in this project show that a 0.5 second delay in real-time display makes it extremely difficult for the teleoperator to control the remote vehicle. This problem is addressed in the paper by using a predictive display (PD) system which provides intermediate updates of the remote vehicle's environment while waiting for actual camera images. The predictive display utilizes estimated positions of the ego vehicle and of other vehicles on the road computed using model-based extended Kalman filters. A crucial result presented in the paper is that vehicle motion models need to be inertial rather than relative and so tracking of other vehicles requires accurate localization of the ego vehicle itself. An experimental study using 5 human teleoperators is conducted to compare teleoperation performance with and without predictive display. A 0.5 second time-delay in camera images makes it impossible to control the vehicle to stay in its lane on curved roads, but the use of the developed predictive display system enables safe remote vehicle control with almost as accurate a performance as the delay-free case. 

This paper develops a cost-effective vehicle detection and tracking system based on fusion of a 2-D LIDAR and a monocular camera to protect electric micromobility devices, especially e-scooters, by predicting the real- time danger of a car- scooter collision. The cost and size disadvantages of 3-D LIDAR sensors make them an unsuitable choice for micromobility devices. Therefore, a 2-D RPLIDAR Mapper sensor is used. Although low-cost, this sensor comes with major shortcomings such as the narrow vertical field of view and its low density of data points. Due to these factors, the sensor does not have a robust output in outdoor applications, and the measurements keep jumping and sliding on the vehicle surface. To improve the performance of the LIDAR, a single monocular camera is fused with the LIDAR data not only to detect vehicles, but also to separately detect the front and side of a target vehicle and to find its corner. It is shown that this corner detection method is more accurate than strategies that are only based on the LIDAR data. The corner measurements are used in a high-gain observer to estimate the location, velocity, and orientation of the target vehicle. The developed system is implemented on a Ninebot e-scooter platform, and multiple experiments are performed to evaluate the performance of the algorithm.